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Cs| We consider the dynamics, existence and stability of the equihbrium states for large 

<0 populations of individuals who can play various types of non-cooperative games. The 

(N 

5h players imitate the most attractive strategies, and the choice is motivated not only by 

^> the material payoffs of the strategies, but also by their popularity in the population. 

"^ The parameter which determines the weights of both factors in the equilibrium states 

^ has the same analytical form for all types of considered games, and is identified with 

the sensitivity to reinforcements parameter in the Hernstein's Matching Law. We 



o 

r^ prove theorems of existence and uniqueness, and discuss examples of multiple locally 



stable polymorphic equilibria for the considered types of games. 



^ I. INTRODUCTION 

^ In the evolutionary game theory the theoretical approach to model the dynamics of popula- 

Q tions is based on the proportional fitness rule. If the random matching and imitation of the 



(N 



X 



individuals with highest payoffs are assumed, the equations which govern the evolution of 
such populations are the celebrated replicator equations, cf. e.g. [IHS], and references cited 
therein. 

However, in general the individuals can be oriented not only toward imitating the highest 
payoff strategies, but they can also take into account other, "noneconomic" factors, in par- 
ticular popularity of strategies in the population. The idea of combining together the bias 
towards imitating the strategies of the most payoff-successful agents and the strategies of 
the majority is not new, cf. for example [7], where the ideas of imitating the successful, and 
copying the majority (the conformist transmission), are put together to stabilize the coop- 
eration in populations of individuals, see also [S] for another argument that noneconomic 
factors influence human's behavior. 



We consider a theory of evolution of social systems, that generalizes the standard propor- 
tional fitness rule of the evolutionary game theory. The biological fitness of a behavior, 
strategy, measured by its payoff from interactions, is replaced by more general function, 
the attractiveness of the behavior. The attractiveness of the strategy is assumed to depend 
not only on its payoff, but also on its actual popularity (fraction) in the population. The 
parameters of the attractiveness function describe different psychological characters of the 
members of the population. We consider the model based on the generalized Cobb-Douglas 
utility function, cf. [Hj, jTU], [H]- We find a parameter that describes different personality 
profiles of the players, and that can be identified with the sensitivity to reinforcement in 
the Matching Law of mathematical psychology |J^. In our setting it determines stability 
of polymorphic equilibria in all considered classes of games. We note that such equilibria 
for games with two strategies played in infinite populations were found for example in the 
aspiration-based models, cf. for example [T3|[T^. in general multi-person games, cf. [15], in 
the models of social dilemmas with synergy and discounting, cf. jTB], in the Stag-Hunt multi- 
person games, cf. for example [T7], in the multi-person Snowdrift game, cf. for example |18| 
and references cited therein. 

We prove theorems of the existence of polymorphic equilibria, and identify sufficient con- 
ditions for their uniqueness for particular classes of the considered games. We also find 
examples of the existence of more than one (locally) stable polymorphism. 
In the next section we formulate the general model and discuss its basic properties. In 
section III we briefly remind, for the convenience of the reader, an existence and uniqueness 
theorem for a general class of two-person symmetric games with two strategies, played in 
populations with social and material preferences, proved in ^llj. We also provide examples 
of multiple stable internal equilibria for two-person symmetric games with three strategies. 
In section IV we discuss polymorphic equilibria for two-person asymmetric games with two 
strategies, and in section V we consider the multi-person games, in particular the Public 
Good game. In section VI we conclude and discuss some open problems. 

II. MODEL 

We consider an inflnite homogeneous population of individuals who interact through a 
random matching, playing at each instant of time a two-person or a multi-person non- 



cooperative game. The players have a finite number K of behavioral types (strategies). Let 
Ni{t) denotes the number of individuals playing the strategy z, N = Ni + N2 + ...Nk- 
the fixed size of the system, pi = Ni/N -the frequency, or popularity of behavior i for 

2 = l,2,...,i^. 

The players who play a strategy i review their strategy according to the Poisson process 
with the arrival rate r^. We model the corresponding stochastic processes as a deterministic 
fiow. The balance conservation equations read, cf. e.g. |5], section 4.4: 

Pi{t) = ^[Pj^jP] - Pi^iPl], i = 1, -K, (1) 

where p* is the probability that the agent playing the j strategy will switch to the i strategy. 
We assume that p* is proportional to the attractiveness Ui of the strategy i: 

P] = cui. (2) 

In general the attractiveness of a strategy can be a complicated function of various factors, 
describing the state of the system and the characteristics of the players. Many social and 
biological interactions are based on imitation processes, where individuals adopt more suc- 
cessful and more popular strategies with larger probability than less successful and less pop- 
ular ones. We choose for the attractiveness of the strategy i the generalized Cobb-Douglas 
utility function, cf. P-ITT]: 

u,{t)=p]-''u]-^, t = l,...,K, (3) 



with {a, j3) E [0,1] x [0,1]. The formula pi) states that the attractiveness of a strategy 
depends not only on the payoff of the strategy, but also on its actual popularity in the 
population. The parameters a, /3 determine the responsiveness of the function Ui to changes 
of the current popularity and of the mean payoff of the action i. They define different social 
and material preferences of the individuals, in other words their different personality profiles. 
More attractive strategies ought to have an evolutionary advantage in the considered social 
systems. Note that the attractiveness of the strategy i is increasing and concave function 
of both arguments, i.e. of the mean payoff Ui and of the popularity pi in the population. 
In particular when the attractiveness reaches a higher level, the changes are slower. The 
attractiveness of the strategy becomes zero if its mean payoff or its popularity in the pop- 
ulation is zero. In the Appendix, p. A we define and characterize the ideal types of the 



personality profiles for a,/3 G {0,1}, and characterize their basic properties, cf. [T0| [TT] . 
All other values of the parameters a, /3 describe intermediate personality profiles. We show 
below that the combination of these parameters 

plays a crucial role in determining the polymorphic equlibria and their stability, and can be 

identified with the sensitivity parameter which links the relative rates of reinforcements and 

responses in the Hernstein's Matching Law [12j. 

As postulated in ([2]), the strategies with higher attractivenesses have larger probability to 

be imitated. Assuming that the arrival rates rj are constants, and rescaling the time we 

obtain the system of equations which has the formal structure analogous to the replicator 

equations: 

K 

Pi{t) =u{— -pi), u:=^Uj, z = l,2,...,K. (5) 

i=i 
Thus, the fraction pi of strategy i increases if its normalized attractiveness — G [0, 1] is 

bigger than the actual fraction of the strategy i, and decreases if it is smaller. In particular, 

for a = /3 = 0, corresponding to Homo Afectualis (cf. the Appendix, p. A) the evolution 

equations ([s]) are identical to the replicator equations of the evolutionary game theory. 

All critical points of the dynamics ([s]) are obtained as solutions of the system of i^ — 1 

algebraic equations 

— = — = ... = — . 6) 

Pi P2 Pk 

which, after substituting (IS]) is equivalent to 

Pi \^ij 
where z/j is the mean payoff of strategy i. In particular the stability properties of the 
solutions of eqs. ^ depend on the combination s of the parameters a, /3, which characterize 
the personality profile of the players, rather than separately on each of them. The sensitivity 
parameter s plays an important role in the matching law in the operant response theory of the 
mathematical psychology, in particular as a measure of the degree to which, in equilibrium, 
the response ratio changes when the reinforcement ratio is modified, cf. for example [T01 - IT2] . 
and references cited therein. 



III. EQUILIBRIA FOR TWO-PERSON SYMMETRIC GAMES 

In this section we consider populations which play symmetric 2-person games. For a con- 
venience of the reader we first remind the results for the case of two strategies, proved in 

m 

A. 2-Person Games with 2 Strategies 

For the symmetric 2-person games with two strategies (denoted 1,2), with the payoff matrix 

1 2 
lab (7) 

2 c d 

where a, b, c, d are arbitrary positive numbers, we obtain the full characterization of the 
equilibria. With the normalization condition pi + P2 = 1 eq. ([s]) reduces to the evolution 
equation 

p, = {i- p,y-y\-^[ui-^{i - p,r - ^2"V], (8) 

with Ui = {a — b)pi + S, 1^2 = {c — d)pi + d. For each personality profile (a, /3) G [0, 1] x [0, 1] 
there exist two pure equilibria of (pi): pi = and pi = 1 [for a = 1 we put Ui{pi = 0) = 

0, i = 1, 2]. The equilibria of eq. dsl) in which each strategy has a non zero frequency will 
be called Mixed Equilibria, and denoted ME. Each ME corresponds to a fixed point of 
eq. (Is]). For the symmetric 2-person games the following theorem is true: 

Theorem 1. 

For the payoff matrix [^ with positive entries: 

1. For each < s < 00 there exists at least one ME - the fixed point of the evolution equation 

&■ 

II. Denote 5 := (1 - s)^ + (1 + s)^, A := 5^ - 4^. If 1. A < 0, or 2. B > 0, or 3. 
be > ad, then ME is unique. 

III. For each < s < 00 there exist at most three ME. 

IV. There exist three ME iff A > 0, B < 0, and U{zi)U{z2) < 0, where Zi^2 ■= [-5^v^]/2, 

cz+d 
'az+b' 



and U{z) := Inz + s In^^^, z > 0. 
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V. If a ME is unique, then it is globally stable in (0, 1) under the considered dynamics. If 
there are three ME, then the middle one is (locally) instable, the other two stable. 

The statement I implies that the dependence of the attractiveness of a strategy on its 
popularity implies the existence of at least one ME for all symmetric two-person games with 
positive payoffs, including all types of the social dilemma games. In particular this "solves 
the dilemma" of the Prisoner's Dilemma population game. The statement II. 2 implies that 
for s < 1 the cooperation level is unique, and does not depend on the initial distribution 
of strategies. For the proof, interpretations and applications to various types of 2-person 
games the reader is referred to |11) . 

B. 2-Person Symmetric Games with 3 Strategies 

1. General results 

2-person symmetric games with 3 strategies play important role in the population game 

theory. First we formulate the general setting in the frame of populations with complex 

personality profiles, then we discuss particular types of such games. 

The general symmetric 2-person game with 3 strategies, denoted 1, 2, 3, has the payoff 

matrix 

12 3 



Oil ai2 ai3 



021 0-22 C(-23 
^31 032 033 

where we assume aij > V2,j G {1,2,3}. The mean payoffs of strategies 1,2,3 are defined 



as 



z/,(t) = 5^ai,p,(t), ^ = 1,2,3, (10) 

with ps = 1 — pi — P2- The dynamical system (|5| has the form 

Pi = p\-%l-Vi>\-^-Vi[p\-''i^l-^ + (P3)'-"^l~^], (11) 

P2 = P\-%1 - P2yf^ - P2[pY''vI-^ + {Pzf-'-l^l-^ (12) 



7 
For the 3-strategy games we define mixed equilibria (ME) of ( 11 ) as tlie critical points of the 



above dynamical system with all nonzero coordinates Pi, i = 1,2,3. We shall alternatively 
use the term polymorphic equilibria. With notation 

x:='5. y:='5 (13) 

Pi Pi 

the equations for ME can be written, using ([s]), (|6]), (10), in the form 

^^(a^^ + a^^x + a^Y^ (14) 

Van + ai2X + ai3yJ 

_ f asi + a32X + a33y Y /icr\ 

\an + ai2X + aisy J 
The frequencies of the strategies in ME are obtained from the reverse formulas: 

Pi = z—^ , — , P2 = Pix, P3 = Piy. (16) 

1 + X + y 

In general the number of ME and their stability depend on the payoff matrix and on the 

sensitivity parameter s. 

One of the most celebrated 3-strategy games is the Rock-Paper-Scissor (RPS) game, cf. for 

example [HI E]- The population of players with complex personality profiles, playing the 

RSP game has been investigated in [19j . In particular, in the "standard" RPS game the 

critical point (|, |, |) is asymptotically stable for all a, /3 G (0,1]. For other properties 

of the solutions, in particular the existence of limit cycles, the Hopf and the Boutin-like 

bifurcations, the reader is referred to |19| . 

Below we analyze other important classes of 3-strategy symmetric games, in particular the 

coordination games, for which there exist multiple stable polymorphic equilibria, and the 

iterated Prisoner's Dilemma game with three strategies AllD, AUG, TFT, with unique stable 

ME. 



2. Coordination Game 
We consider the coordination game with 3 strategies and the payoff matrix 





1 2 3 


1 


2 1 1 


2 


1 3 1 


3 


1 1 3 



(17) 



8 



We demonstrate the existence of three ME for s = 3. For s = 1 the system (14), (15) can be 
solved exphcitly. The unique nonnegative solution is x = y = ^(1 + V^), which, according 
to (16), gives the equilibrium state {pi,P2,P3) with the frequencies pi = 2 — y/S, P2 = Ps = 



UVs-i). 



For s = 2 the system (14), (15) is of the third order. In order to obtain analytical solution 



we subtract (15) from (14), and obtain the equation 



(x - y){2 + x + yf = 4(x - y){l + 2x + 2y). 



(18) 



It can be easily shown that (18) has positive solution only if y = x, in which case the solution 



of the system (14), (15) is the pair (xo,a;o), where xq = 2.45 is the unique positive root of 
the polynomial 



F{x) = Ax^ - 8x^ -Ax-1. (19) 

The corresponding unique equilibrium state reads {pi,P2,P3) — (0.17,0.415,0.415). When 
we farther increase the value of the sensitivity parameter s the uniqueness is lost. In Figure [T] 
we show time evolution of frequencies {pi,p2, Ps) for two values of the sensitivity parameter s 
for particular initial data, symmetric with respect to piQ (i.e. for p20 = Pso)- For s = 2.15 the 
equilibrium is unique (not shown). For a value s > sq E (2.15,2.30) there emerge multiple 
equilibria. For s = 3 (left diagram), and s = 3.75 (right diagram) there are three ME, two of 
them stable, one unstable. In each diagram the frequencies reach asymptotically one of the 
stable ME. Note the influence of the unstable equilibrium in the early stage of the evolution. 
First the trajectory approaches the unstable symmetric ME {pi,P2,P3) — (0.14,0.43,0.43), 
then it tends to the locally stable equilibrium: {pi,P2,P3) — (0.126,0.225,0.649). The other 
"symmetric" locally stable ME is {pi,P2,P3) — (0.126,0.649,0.225). The amount of time the 
trajectory stays in a neighborhood of the unstable equilibrium depends on the sensitivity 
parameter s, and on the personality profile parameters a, /3. For s = 3.75 both locally stable 
ME tend to the relevant vortexes of the game simplex, cf. the right diagram in Figure [T| 
We explain the emergence of the multiple equilibria for s = 3. First we look for the symmetric 



solution p2 = P3, i.e. x = y. The system (14), (15) reduces to one equation 



8x'^ - AOx-^ - 24x^ - 4a; - 1 = 0. 



(20) 



The unique positive root of the polynomial in (20) is x = 5.556. The corresponding equi- 
librium reads {pi,P2,P3) — (0.082,0.459,0.459). There are two other, symmetric ME. 



The first zero gives the ME: (^1,^2,^3) — (0.047,0.057,0.896). As expected from the 
symmetry of the second and third strategy, the second zero gives the "symmetric" ME: 
{pi,P2,P3) — (0.047,0.896,0.057). There are also four "partially mixed" equilibria: two on 
the edge pi = 0, and one on each other edges, in the neighborhood of the corners 2 and 
3, i.e. near p2 = 1 and p^ = 1. For increasing values of the sensitivity parameter s the 
stable equilibria approach the relevant edges of the game simplex. In the Appendix, p. B we 
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FIG. 1: Time evolution of frequencies Pi,P2,P3 for the coordination game (17). Left: s = 3, right: 
s = 3.75. 



prove the existence and uniqueness theorem for another class of the asymmetric coordination 
games, in which both players are better off if they play different strategies (anti-coordination 
games) . 

3. Repeated Prisoner's Dilemma Game 



Another interesting example of three-strategy games is the 2-person, 3-strategy finitely 
repeated Prisoner's Dilemma Game with the payoff matrix 

AllC AllD TFT 

AllC Rm Sm Rm , , 

(21) 

AllD Tm Pm T + P{m - 1) 

TFT Rm S + P{m - 1) Rm 

where T > R> P > S are the payoffs in the one-shot PD game, AllC (AllD) is the strategy: 
play always C (play always D), TFT is the strategy Tit for Tat, and m is the number of 
rounds. 
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In the classical replicator dynamics, with the payoff from TFT additionally reduced by a 
small value (the cost of playing TFT) it has been shown in [21], that the population evolves 
through the cycles of cooperation and defection. 



For the general matrix (21) we checked numerically the existence and uniqueness of ME 



for for wide ranges of the payoffs T > R > P > S and of the parameters m, s. Moreover, 
the share (sum of the frequencies) of the cooperative strategies AllC and TFT decreases for 
increasing ratio ^ and increasing number of rounds, whereas for increasing m it increases, 
in agreement with intuition. 
For the Weak Prisoner's Dilemma (P = 5 = 0) we prove 



Corollary 2. 

There exists an unique ME: {pi,P2,P3) — V2+ 



h 2?^' 2T^)' ^^^^^ ^ = [^S^]'' «/ ^^^ aa^^^ 



(21) with P = S = 0, for all nonnegative sensitivity parameters s. 

--^ — -'^^ and the result follows 

D 



Proof. Eq. (15) for the ME gives y = 1. Eq. (14) gives a = [ 



2m 



from (16). 



Note that for the increasing ratio ^ the shares of the AllC and TFT players in ME decrease. 



whereas for increasing m they increase, as in the case of the general payoff matrix (21 ). 



IV. 2-PERSON ASYMMETRIC GAMES 



A. General Results 



In this section we consider 2-person games played between members of two populations with 
(in general) different personality profiles. The members of both populations choose between 
two strategies A and B. The payoff matrix reads 



(ai,a2) (6i,&2) 
(Ci,C2) {di,d2] 



(22) 



with nonnegative entries. In general the strategies in both populations may be different 
meanings. For simplicity we do not use different notation for the relevant pairs of strategies. 
The row players belong to the population i = I, and the column ones to i = 2. We denote 
Xi, i = 1,2, the fraction of population i which plays strategy A. 
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Let (ftj, Pi) describe the personality profile of the individuals of population i, i = 1.2. The 
members of each population may put different weights to the payoffs and popularities of the 
available strategies in the imitation process. 
We define M*-the attractiveness of strategy j G {A, B} in population i G {1, 2}: 

< = ^1-"'^m'\ (23) 

Ub = (1-^.)'""'47'N (24) 

where Uji is the mean payoff from strategy j in population i: 

vai = aiX2+bi{l-X2), vbi = CiX2+di{l-X2), va2 = a2a;i+C2(l-Xi), 1^32 = b2Xi+d2{l-Xi). 

Thus, the attractiveness of each strategy of a population depends on two factors: the "so- 
cial" one, represented by the fraction, popularity of the strategy in the population, and the 
"economic" one, represented by the mean payoff of this strategy in the considered popula- 
tion. Note that the social factor depends on the composition of the considered population, 
whereas the economic one depends on the composition of the second population. This can be 
interpreted in the following way: the members of each population determine their strategy 
choice observing popularity of the available strategies in their own population, and their 
payoffs from the interactions with the members of the other population. 
The dynamics (IS]), written for asymmetric games, reads 

xi = (l-xi)x}-"VJ,7^^-xi(l-xi)i--i47^S 



^2 = (1 - X2)xl"''u\~^' - X2(l - X2y-'''iy^'^' 



(25) 

'2 

B2 



Note that for ai = (3i = 0,i = 1,2 the system (25) reduces to the replicator dynamics for 



asymmetric games, cf. for example O [6|. The pairs of (xi,X2): (0, 0), (0, 1), (1,0), (1, 1) 



are equilibria in pure strategies of (25). They correspond to the situations in which each 
population plays only one of two available strategies. We are looking for the mixed equilibria 
(ME) in which both strategies have nonzero frequencies for each population. 



With the substitution Zi = jz^, i = 1,2 the system reads 

Z2 = zl-'-^l + Z2r-U'A2^' - Z2(l + ^2)"V^2^^ 



(26) 



From (26) we obtain the algebraic equations for ME: 



_ , a-iZ2+bx \ ^ _ / 0221+62 ^ (07\ 
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where 

s. = ^^, z = l,2 (28) 

denotes sensitivity of players in population i. Thus, for the asymmetric 2-person games 
the ME are determined by relevant combinations of parameters Si,S2 which describe the 
personalities of players respectively in the first and the in the second population. In the 
next subsections we show the existence of ME for typical asjTiimetric games. Here we prove 
that, contrary to the replicator equations for asymmetric 2-person games, the solutions can 
not be periodic. 



Theorem 3. The system (26) does not have periodic solutions. 



Proof. We define the function (j){zi, Z2) = [zi{l + Zi)°'^Z2{l + 2:2)"^] ^, and note that 

g(#l) _^ ^(#2) ^ _«lZ^^!:^Ml!^ _ «2^2'^"^^'^'^a/^ ^ Q 
dZi dz2 Z2{l + Z2Y^ Zi{l + Zi)^^ 

for Zi,Z2 > 0. Applying the criterion Dulac-Bendixson we obtain the thesis. D 

B. Asymmetric coordination games 

1. Pure coordination games 

We consider the model defined in the previous section for the asymmetric coordination games 
with the payoff matrix 

(01,02) (0,0) 

(0,0) {di,d2) 

and Oj 7^ 0, rfj 7^ 0, i = 1,2, played between two populations with different personality pro- 
files. Such games will be called pure (asymmetric) coordination games. We can for example 
think about two populations of different sexes, i = 1 and i = 2 denote respectively men's 
and woman's populations. For ai > di, 02 < (^2 this corresponds to the well known Battle of 
Sexes game. Players from each population evaluate the available strategies according to their 
attractiveness, calculated for each of two populations i = 1,2 with the personality profiles 



(29) 



{ai,j3i), according to formulas (23). With notation (28) we prove the following theorem 
Theorem 4. 
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If SiS2 7^ 1 then there exists a unique ME of the dynamics (26) for the populations with the 



payoff matrices (29): 



Zl 



aia'^2 
didl' 



Z2 



al^a2 
d\'d2 



(30) 



The equilibrium (30) is locally asymptotically stable if S1S2 < 1, and unstable if S1S2 > 1- 



If S1S2 = 1 then for {ai/hiy'^a^2 7^ 1 there are no ME, for [ai/hi] 



51/7*2 



1 there exists the 



family of ME z\ 



Z2 = 0*^(02/62)*^; where c is an arbitrary positive constant. 



Proof: cf. the Appendix, p. C Numerical simulations indicate that if the equilibrium (30) is 



locally stable then it attracts all investigated trajectories from the interior of the simplex of 
the game. We note that the replicator dynamics for the considered asymmetric games (which 
formally coresponds to infinite sensitivities in our dynamics, has an unique ME, which is 
Lapunov stable. In Appendix, p. D, we prove analogous existence and uniqueness theorem 
for another class of asymmetric games (anti-coordination games), in which the players are 
better off if they use different strategies. 

2. Coordination games with multiple ME 



In asymmetric coordination games the positivity of payoffs plays crucial role in the unique- 
ness problem, and in determining the polymorphic equilibria. In the pure coordination 
games considered above the ME is unique, whereas in general coordination games multiple 
stable polymorphic equilibria are possible. For example, the 2-person asymmetric game with 
the payoff matrix 

'(3,2) (1,1) 
(1,1) (2,3) 
has, for Si = S2 = 3, three ME, corresponding to (xi ~ 0.92, X2 ~ 0.82), (xi = 0.6, X2 



(31) 



0.40), and (xi ~ 0.18, X2 ~ 0.08), as can be checked solving (27). Two equilibria are locally 
stable, one unstable, cf. Fig. |2} 
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FIG. 2: Examples of trajectories and critical points for the asymmetric coordination game (31): 
a\ = di = 3,02 = d\ = 2,bi = Ci = l,i = 1,2. All six displayed trajectories start from a 
neighborhood of the points (0, 1) or (1, 0). 



V. MULTI-PERSON GAMES 



A. General results 



In this section we consider multi-person one-shot symmetric games in which each of A^ 
players chooses one of two actions: C or D, C stands for cooperation, D for defection. In 
order to obtain symmetric notation and formulas, we define n := N — 1, and from now on 
we consider the n + 1-person games. This allows to define: 

Ofc : the payoff of player C, when k others play C, 
bk : the payoff of player D, when k others play D, 

where A; G {0, . . . , n}, and a^, bk > 0. 

Let X denote the fraction of the population that plays C. The mean payoffs from both 

strategies are respectively 



^c 



l^D 



$:(;;)x-(i-.r-v. 

k=0 

n 

E 

k=0 



n 



x'^(i-xr-%. 



15 

The evolution equation reads 

X = (1 - x)x^-"z/^"^ - x(l - x)^-"4"'^, (32) 

Substituting z = j^ we obtain 



^ = «'-'lli^) ---"1. (33) 

where W^il^;) = Yl {2)'^kZ^, Wb{z) = ^ (^h^z^^ and f{z) is a positive function which 

A;=0 fc=o 

does not influence the equilibria and their stabilities but only the speed of the evolution. 



Stationary points of the dynamics described by (33) can be obtained solving the equation 



s 

z) ■ 



' \Wb{-z 
where s = -^ is the previously defined sensitivity coefficient. We prove the following 



Theorem 5. 



The dynamics (32) of the symmetric (n+l)-person game has at most 2n + l mixed equilibria. 



Proof. We define the function U{z) : IR+ — > IR+: 

Wa{z) 
U{z) =gln -\nz. 

Wb{z) 

Zeros of U are stationary points of the considered dynamics, and sgn{U{z)\zg) = sgn(i;|2y). 

It is sufficient to show that U{z) = at at most 2n + 1 points in (0, +oo). To this end we 

U'f )= Ek _ Ek_^= ^W'aWbz - sW'sWaz - WaWb 
^^' ^Wa ^Wb z WaWbz 

The polynomial sW'^Wbz — sW'^Waz — WaWb is at most of the order 2n, therefore U'{z) 

has at most 2n zeros. Since between each two zeros of U there has to be a zero of U'{z), the 

function U{z) can have at most 2n + 1 zeros. D 

Theorem 6. 

For the (n+l)-person games with two strategies and with all payoffs positive there exists at 
least one mixed equilibrium. 

Proof. Due to continuity of the considered dynamics is enough to prove that both boundary 
equilibrium points corresponding to pore equilibria: x = 0, x = 1 are unstable. This will 
be proved in two lemmas below. D 
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Lemma 1. Let p = min{i: «« > 0}, r = min {i: hi > 0}. // 



ps > rs + 1, or 
ps = rs + 1 and I J Op < I ) 6^, 

then the point x = Q is asymptotically stable. If 



(34) 
(35) 



ps < rs + 1, or 



ps = rs + 1 and ( ) Op > 



n 



n 



then the point x = zs unstable. 



Proof. We note that 



Wb{z) 



apzP + 0{zP), 



We obtain 



for the condition (IMl): WrciUiz) = hmln ( -— r — 



i;w 



for the condition (35): \im.U(z) = In , , , 

^ .^0 ^ ^ I {^)br 



<0. 



-CXD, 



In our one dimensional system the stationary point z > is asymptotically stable iff 
3£>o z\(z,2+e) < A z\(^z-e,z) > 0. The sufficient condition is U'{z) < 0. On the contrary, if 
U'{z) > 0, then the point z is unstable. Analogously, the sufficient condition for stability of 



z = reads: MuIz^q U{z) < 0. Since sgn{U{zQ)) = sgn(i;|^Q), then in the cases (34) and (35) 
we have i < for a certain Sz > and z G (O,^^). From the equality x = fn^ it follows, 
that then for a certain Ex > and x G (0, Ex) we have a; < 0. 
The proof of the second part of Lemma [T] is analogous. D 

Lemma 2. Let p' = raa,x{i: ai > 0}, r' = max{i: bi > 0}. // 



p's > r's + 1, or 
p's = r's + 1 and 



api > 



n 



K', 
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then the point x = 1 is asymptotically stable. If 

p's < r's + 1, or 

p's = r's + 1 and I Ictp' < I , )&r', 
\Pj \r'J 

then the point x = 1 is unstable. 

Proof. It is enough to apply Lemma [T] with the strategies interchanged. D 

The above theorem guarantees the existence, however not uniqueness of the ME (we remind 
that for the 2-person symmetric games with positive payoffs the uniqueness has been proved 
for s < 1. Below we give an example of a 3-person game with positive payoffs and three ME 
for the sensitivity s = 1. 

Example 

Let qq = 5, ai = a2 = 1, 6o = 1, &i = f, ^2 = 5|. The corresponding payoff matrix reads: 





11 


12 22 


1 


5 


1 1 


2 


1 


2 c;i 

3 ^3 



(36) 



Note that this game can be treated as a coordination game (the players are better off if they 
play the same strategies). The equation for ME reads: 



aQZ + aiz + 02 
boz'^ + biz + 62 



(37) 



which, for s = \ has three positive roots, resulting, after substitution z = ^3^ in three ME: 
Xi = |, X2 = |, ,Xi = |. We checked that xi and X3 are locally stable, and X2 unstable. 

B. Public Goods game 

We apply the general results of the previous subsection to the important {n + l)-person 
game-the Public Goods (PG) game. In this game each of (n+ 1) players receives an amount 
g, and chooses one of two actions: C: Contribute with g into the common pool or D: Do 
not contribute. Let k among n + 1 players choose C. The amount kg in the common pool 
is multiplied byr, n + 1 > r > 1, and distributed equally among n + 1 players. Using the 
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notation for the payoffs in the (n+l)-person game introduced in (V A) and the normahzation 
g = 1 we obtain the following payoffs of the strategies respectively C and D in the PG game: 

r 1 



ak = {k + l)p, bk = kp + l, p:-- 



<p <1. 



(38) 



n + 1 n+ 1 

Below we investigate equilibria for the PG game. We show that their number and stability 
properties are the same as of a 2-person Prisoner's Dilemma (PD) game, of which it is the 
multiple, see definition below. Thus, since any 2-person PD with positive payoffs has at 
least one and at most three internal equilibria, the same is true for the PG game. We begin 
with the definition of the multiple of any 2-person symmetric game. 

Definition 1. Multiple of the 2-person symmetric game with the payoff matrix 

a b 
c d 

is the {n + l) -person symmetric game with the payoffs 

ttk = ka + {n — k)b, 
bk = kc+ {n — k)d. 

k = 0, 1, ..., n. Thus, in the multiple of the 2-person symmetric game the payoff of a player 
is the sum of his payoffs from all the 2-person games with the other n players. We prove 

Lemma 3. Any 2-person symmetric game and its multiple have the same equilibria in the 



dynamics (32). Moreover, the stability properties of the equilibria of both games are the 



same. 



Proof. For the multiple of the 2-person game we have 



WAiz) = J2 (^) (^« + (^ - k)b)^'' = J2 it) ^^"^ -b)z' + nJ2 Q bz' 

fc=0 ^ ^ fc=0 ^ ^ k=0 ^ ^ 



n(a 



b)^J2i'^^ ]]z''~' + nb{l + z)^ 



fc=i 



k-1 



n{a - b)z{l + zY~^ + nb{\ + zf = n{l + z)''-\z{a - 6) + (1 + z)b) 



n{l + z)''-\za + b) 
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and analogously Wb{z) = n{l + z)^~^{zc + d). As previously we define the function U, zeros 
of which are stationary points of the considered dynamics: 

U(z) = s In ^^^ , , — In 2; = s In I | —\nz. 

^ ' Wb{z) \zc + dj 

Thus, the function U{z) for the (n + l)-person game which is the multiple of the 2-person 
game is the same as for the 2-person game. In consequence both games have the same 
equilibria, and the stability properties of the corresponding equilibria are identical. D 

Now we come back to the PG (n+l)-person game with the payoffs at = {k + l)p, bk = kp+1, 
with ^^ < p < 1. We note that they can be rewritten as 

Ofc = kp \- (n — k)p—, 

n n 

, pn + 1 , , , 1 

bk = k \-in-k)-. 

n n 

It means that Public Goods game is the multiple of the PD game with the payoff matrix 

(39) 



p(n+l) 


P 
n 


n 


pn+l 
n 


I 
n 



therefore, from Lemma 3 it is sufficient to investigate the dynamics for the PD game (39). 



VI. CONCLUSIONS 

In this work we discussed the evolutionary dynamics of populations of agents with complex 
personality profiles, which is governed by the attractiveness of strategies rather than by their 
payoffs. The agents can play various types of non-cooperative games, including general two- 
person asymmetric games and multi-person games. The profiles determine the weights the 
agents associate to the payoffs and the popularities of the available strategies. It turns out 
that the polymorphic equilibria and their stability properties are in general determined by 
a single sensitivity to reinforcement parameter which characterizes the population. The 
parameter has an interpretation in the Matching Law of mathematical psychology. For 
general characters there exist stable mixed equilibria of the considered dynamics, not present 
in the classical evolutionary game theory approach based on the replicator dynamics. 
There are various interesting open problems related to the presented research: generaliza- 
tion for systems with more behavioral types, introduction of other, more general types of 
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attractiveness functions, which for example take into account the speed of the changes of the 
actual composition of population. It will also be interesting to allow the actors to change 
their personalities during the interactions and/or to react with delay to received impulses 
(information lag). In particular, the delay can be present only in the social or only in the 
material part of the attractiveness fuction. Models with a finite number of heterogeneous 
agents with different personality characteristics seem to be another interesting area of future 
research. 
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VII. APPENDIX 

A. Ideal personality profiles 

HE (Homo Economicus): a = 1, /3 = 0. It means that HE assesses attractiveness of the 

action exclusively through its effectiveness (mj = Ui). HE is interested exclusively in the 

future prospects, 

HS (Homo Sociologicus): a = 0, /3 = 1. The HS assesses attractiveness of the action only 

through its popularity, the past effect for him/her {ui = pi). HS is insensitive to the payoffs 

of the game. 

HT (Homo Transcendentalis): a = /3 = 1. It is an ideal type, for which every action has the 

same attractiveness (mj = 1). It describes a personality not interested in the effectiveness of 

a behavior or in its propensity, but rather by some other values. Thus, HT is the ideal type 

insensitive to the payoffs and the popularities of the strategies. 

HA (Homo Afectualis): a = /3 = 0. The attractiveness function takes the form Ui = UiPi 

The corresponding evolution equations reduce to the standard replicator equations. Inserting 

a = into (IS]) we obtain a generalized form of replicator equations, in which the material 

payoffs are included in nonlinear way: 

P^=P^ E PA^I'^-^Y^ l = l,...K. (40) 

j=l,...K 

In particular for /3 = we obtain the usual replicator equations for the two-person symmetric 
games with K strategies. 
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B. Symmetric anti-coordination games with 3 strategies 



Here we consider the 3-strategy game with the payoff matrix 





1 


2 


3 


1 





ai 


02 


2 


ai 





as 


3 


^2 


as 






(41) 



aj > 0, ^ = 1, 2, 3, in which the players are better off if they use different strategies, then 
otherwise. We prove 



Corollary 7. There exists the unique ME (^, |, ^) of the game (41) with a, = a > 0, i 



1,2,3 for alls > 0. 



Proof. We check that x = y = 1, which corresponds to the above ME, satisfy (14), (15). 



For X y^ y, subtracting (15) from (14) we obtain the equation 

{i + yy-{i + xy 

X ~ y = r , 

{x + y}' 

which can not be satisfied due to different signs of both sides. 



D 



For the general anti-coordination game with arbitrary aij, s such that Cjj > akk > for all 
k and all i ^ j, i,j, k = 1,2, 3, we checked numerically the existence of an unique ME for 
all considered numerical values of these parameters. 

C. Asymmetric pure coordination games 



Proof. The dynamics (26) for the pure coordination game 



(01,02) (0,0) 
(0,0) id,,d2) 



with a,- > 0, dj > 0, i = 1,2, reads 



l-/3i 



Zl 



r^^^i + z,r^ (^] ' -z,{i + z^r^ ( -^ 



\1+Z2j 



Z2 — Z2 



l-fe 



;i+^2)"^fe)' "'-^2(1+^2)"^(t^ 



l-/3i 
1+22 J ' 

1-/32 



1+zi 



(42) 



(43) 
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The equations (|27D for ME of (|43|) read: Zi = [fz2 



Z2 



d2 



Zl 



Linearization of 



(43) around the equihbriuni (30) leads to the matrix 



i-/3i 



-ai(l + ^i)"^ 4^ 



(l-(3,)zl-^il + Z2r^z-^^(^^ 



ii-p,)-zr^ii + -z,r-z^'^i^^ 



l-fe 



-0:2(1 + Z2 



y \a2[ d2 

1 + 21 



l-/3i 



1-/32 



with a negative trace. Using ( p30| ) we show that if S1S2 < 1 then the determinant of the 
matrix is positive, therefore both eigenvalues have negative real parts, i.e. the equilibrium 
(^1,^2) is locally asymptotically stable. Analogously, if S1S2 > 1, then the determinant is 
negative, therefore at least one eigenvalue has positive real part, i.e. the equilibrium {zi, Z2) 
is unstable. The proof of other statements is omitted. D 



D. Asymmetric anti coordination games 



(44) 



We apply the general model for the asymmetric anti-coordination game with the payoff 

matrix 

"" (0,0) (61,62) 
(C1,C2) (0,0) 
and bi > 0, Cj > 0, i = 1, 2. We prove the following theorem 

Theorem 8. 



If S1S2 7^ 1 then there exists a unique polymorphic equilibrium of the dynamics (26) for the 



populations with the payoff matrices (44) ■ 



Zl 



bic'i\^^^ 



c\'b. 



If ^1^2 = 1 then we have the same situation as in Theorem^ with obvious changes of the 
payoff parameters. 



The equilibrium (45) is locally asymptotically stable if S1S2 < 1, and unstable if S1S2 > 1- 



The proof is analogous as in Theorem |4] and is omitted. Numerical simulations indicate that 



if the equilibrium (30) is locally stable then it attracts all investigated trajectories from the 



interior of the simplex of the game. 
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E. Asymmetric non-coordination games 

We consider populations which play asymmetric game with the payoff matrices 

(a,0) (0,6) 
(0,c) (d,0) 



(46) 



This game can be considered as a two-population variant of the Matching Pennies game. 



The dynamics (26) for the game (46), with positive payoffs a,b,c,d reads 

l-/3i 



Zi = Z^ 



1 — Cfl 



:i+zir^(^y^^'-z,{i+z^r^' ^ 



\^+Z2 ) 



1+Z2 



Z2 — Z2 



.,1-02 



;i + ^2; 



Cl2 



1 + 21 



l-fe 



^2(1+^2)- (ife 



1-/32 



(47) 



The equations for equilibria have the form -2^1 = (^) \ -2^2 = ( ^ ) , with the solution 



Zl 



Q^(f2 \ 1+S132 



d¥^ 



Z2 



bd''^\ i+'=i'=2 



ca" 



(48) 



We prove that the equilibrium (48) is locally asymptotically stable for all Si,S2 G 



Linearization of (47) around the stationary point (48) leads to the matrix 



-ai(l + Zl 



ai 



d 

1+22 



l-/3i 



1 - /30zr"Hl + zi)"^^2"'^M TTi: 



-(l-/32)z2(l + ^2)"^^r'''( 



1 + 21 



l-ft 



-^2(1 + 2:2) 



1+22 

621 

1+21 



l-/3i 



l-fe 



which has negative trace and positive determinant. 



